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ABSTRACT 

This report describes a project which developed and 
implemented a process for the ongoing identification of national 
information needed for improving programs and services to infants, 
toddlers, children, and youth with disabilities* The process involved 
creating a polling panel of stakeholder groups, creating a list of 
national program information needs for use in snrveying the 17 
members of the polling panel, and conducting a two-round Delphi 
process to rank the relative importance of the information needs. The 
results of the process indicated that the polling panel members 
perceived the following areas to be the most critical areas of 
information need: personnel development, the role of special 
education in reform, accountability for outcomes, and interagency 
collaboration. Results indicate a high degree of need for information 
that can guide state and local decisionmakers responsible for 
implementing policies and practices that will improve the results of 
education for children and youth with disabilities. Appendixes 
contain a list of polling panel members and copies of materials sent 
to polling panel members. (JDD) 
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Executive Summary 



The identification and tracking of relevant information is critical to decisionmaking 
and policy development in contemporary society. In recognition of this reality, Congress 
amended the IDEA to assure that information would be gathered and analyzed that was 
necessary for achieving program and system improvements at the State and local levels. The 
IDEA directs the Secretary of Education to develop and implement a process for the on- 
going identification of national information needed for improving programs and services to 
infants, toddlers, children, and youth with disabilities. The National Association of State 
Directors of Special Education (NASDSE) under its Project FORUM Contract No. 
HS92015001 with the Office of Special Education Programs (OSEP) has been assisting 
OSEP with this effort. This paper reports the results of the Project's initial efforts to 
identify program improvement inJformation needs. 

The process used to identify program improvement information needs was developed 
during December 1992 with extensive input from an expert consultant and a variety of 
stakeholders invested in the education of students with disabilities. The process involved 
three steps: 

1) Creating a polling panel of stakeholder groups that included parents, advocates 
and professionals with a primary focus on more than one specific disability category, 
experts on the process of transition from school to adult life, local and State school 
board members, and groups representing constituencies with political and fiscal 
authority (e.g., the Council of Chief State School Officers). Individuals were 
identified who have regular and systematic contact with other parents and/or 
professionals who similarly serve the needs of children and youth with disabilities. 

2) Creating a list of national program information needs for use in surveying Polling 
Panel members. Project FORUM staff developed an initial list that was based on 
information needs generated in February 1992, when Project FORUM convened a 
two-day Focus Group of experts to assist OSEP in identifying information needed for 
program improvement under Section 618 of the IDEA. These experts generated 
individual information needs statements and collapsed them into broad categories of 
information needs. 

3) A two- round Delphi process was used to rank the relative importance of the 
broad categories of national program information needs. For the first round, all 
members of the Polling Panel were asked to react to the initial list of information 
needs. Edits and revisions to the list were encouraged. Polling panel members were 
asked to identify the most important information needs as well as indicate those that 
were not of interest to their constituencies. Project FORUM staff analyzed the 
Polling Panel's feedback, synthesized the stakeholder input, and refined the 
information needs statements. During round two. Polling Panel members were asked 
to re-rate the categories of information needed relative to the impact of the 



information need on improving programs for students with disabilities and the 
importance of the information in leveraging program improvement in the not too 
distant future. The ratings from the first round of the Delphi process were also 
included so that Polling Panel members could re-evaluate their ratings based on 
those of other respondents. 



The results of the process indicated that the Polling Panel members perceived 
Personnel Development, Tlie Role of Special Education in Reform, Accountability for 
Outcomes, and Interagency Collaboration to be the most critical areas for national 
information needed for program improvement Respondents commented that all of the 
information needs categories could be ranked as high priorities and that actual ratings 
reflected the relative position of the categories rather than an absolute standard. These are 
initial results of an ongoing process. Project FORUM at NASDSE will continue to build 
upon the State level needs identification efforts of other OSEP sponsored activities and 
projects (e.g., the Regional Resources Centers; the National Information Action Center at 
NASDSE); and obtain information about the needs of other stakeholder groups directly 
&om national organizations that serve these constituencies. 

The results of this initial effort at identifying information needed to improve the 
management, administration, delivery, and effectiveness of special education programs offer 
several suggestions for OSEP's consideration. One of these is investing in activities that will 
assist OSEP's network of knowledge producers to develop vehicles for collaborating with 
local administrators and teachers in order to produce information that is relevant to their 
current information needs. A second strategy is to facilitate the timely and meaningful 
exchange of information between OSEP's knowledge produces and the State and local 
administrators who are in a position to apply this knowledge to improving the education 
programs serving infants, toddlers, children, and youth with disabilities. 
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IDENTIFICATION OF NATIONAL INFORMATION 
NEEDED FOR PROGRAM IMPROVEMENT 



Background and Introduction 

The critical importance of information in decisionmaking and policy development has 
long been recognized, and contemporary technology has made the identification, tracking, 
and analysis of information in a timely fashion a critical necessity for organizations in our 
society. It is impossible for individuals who are responsible for the implementation of the 
Individuals with Disabilities Education Act (IDEA) to access and read all of the ever- 
growing array of information that could be useful in helping them make decisions that would 
improve the system and yield better results for infants, toddlers, children, and youth with 
disabilities. Recognizing the implications of this situation prompted Congress to amend 
Section 618(c)(3) of the IDEA to require the Department of Education to undertake 
activities to "identify implementation issues, desired improvements, and information needed 
by State and local agencies to achieve such improvements ..." 

The statute directs the Secretary of Education to develop and implement "a process 
for the on-going identification of national program information needed for improving the 
management, administration, delivery, and effectiveness of programs and services under the 
Act." The process is to be conducted in cooperation with State Education Agencies (SEAs) 
to insure broad Statewide input. The National Association of State Directors of Special 
Education (NASDSE) under its Project FORUM Contract No. HS92015001 has been 
assisting the Office of Special Education Programs (OSEP) with this effort. This paper 
reports on the results of the Project's initial efforts to identify program improvement 
information needs. 

The process used to identify program improvement information needs was designed 
durir the Fall of 1992. Project FORUM convened a series of three stakeholder meetings 
to obtain input into the design of the information needs identification process. A briefing 
paper outlining the task as well as several considerations and options related to need 
identification was developed by an expert consultant and distributed to stakeholders before 
attending the informal half-day meetings. Telephone interviews were conducted with 
stakeholders who could not attend any of the meetings. A report on the results of this 
activity was submitted to (and subsequently approved by) OSEP at the end of December 
1992. 

As discussed in the December 1992 report, a marketing model was selected as the 
primary approach to information need identification and analysis. The consultant preparing 
the background paper as well as stakeholders who had attended planning meetings 
supported the use of this model ~ particularly favoring the concept of stakeholders as 
"customers." The marketing model involves using the strategy of polling "real" stakeholders, 
not just "experts", to find out what they perceive to be their information needs. This 
approach was considered time and cost effective, if the stakeholders had an interest in the 
outcome of the process and the information being requested. 



Method 

The process of identifying and prioritizing information needed for program 
improvement was initiated in March 1993 with the creation of a polling panel. An initial 
list of information needs was generated and members of the Polling Panel participated in 
two rounds of a Delphi process to prioritize the identified needs. 

Creation of a Polling Panel 

The marketing model relies on using a group or groups of consumers as informants. 
For the purpose of identifying national information needed for improvement in the 
management, administration, delivery, and effectiveness of programs and services for infants, 
toddlers, children, and youth with disabilities, Project FORUM established a polling panel 
to function in this role. 

The process of creating a polling panel involved the idv^ntification of specific 
stakeholder groups and the selection of individuals who would represent the interests of 
these stakeholders. Stakeholder groups identified included parents, advocates and 
professionals with a primary focus on more than one specific disability category, experts on 
the process of transition from school to adult life, local and State school board memb<5rs, 
and groups representing constituencies with political and fiscal authority (e.g., the Council 
of Chief State School Officers). With assistance of the leadership in many national 
stakeholder groups, individuals were identified wh o have regular and systematic contact with 
parents or professionals whose interest in the educational needs of children and youth with 
disabilities are similar. (A list of Polling Panel members is attached as Appendix A.) 

The Polling Panel was used to react to and revise working lists of potential 
information needs for program improvement. Polling Panel members were also involved 
in verifying and prioritizing lists, and some members assisted in collapsing the lists of very 
specific needs into broader categories of inforra/aion. 

Creating an Initial List of National Program Information Needs 

The marketing model requires consumers to react to a 'test product'. In the case of 
this task, the 'test product' was an initial list of national program information needs. There 
are an unlimited number of ways of generating such a list ranging from pure speculation to 
elaborate scientific surveying. Given resources available for this effort plus the relatively 
short time frame for obtaining the information, and in keeping with the advice of 
stakeholders. Project FORUM derived this initial list from existing information sources. 

Project FORUM asked members of the Polling Panel to respond to a list that was 
based on information needs generated in February 1992, when Project FORUM convened 



Identification of National Information Needed for Program Improvement 2 
Projea FORUM at NASDSE • Contract No. HS92015001, Year 1 Deliverable #8 June 30, 1993 



ERIC 0 



a two-day Focus Group of experts to rssist OSEP in identifying information needed for 
program improvement under Section 618 of the IDEA- Focus Group participants 
individually generated lists of issues and identified information need statements related to 
them. Collectively, participants collapsed these individual need statements into broad 
categories of information needs that became incorporated into the first draft of OSEP's 
special studies agenda. The output from this meeting was used as the base for developing 
the 'test product' for the Polling Panel's reactions. 

Other existing information reviewed to generate this initial list included information 
from Project FORUM's Issues Tracking Database, results of the annual survey of State 
Directors of Special Education that is conducted by the National Information Action Center 
at NASDSE, and material supplied by other national associations (e.g., NASBE, CCSSO) 
and national clearinghouses, (e.g., NICHCY, ERIC at CEC) that maintain databases about 
the inquiries they receive. The information gathered essentially reiterated the list of issues 
and topics that had been generated by the experts at the February 1992 Focus Group thus 
confirming the eariier work. The initial list of information needs given to the Polling Panel 
were stated in broad categories and included a description of the kinds of improvements the 
information was needed to achieve. 

Initial List of Information Needs 

• Program Management and Reguiatory Flexibility. Currently, many stakeholders express 
interest in information about how regulations may "enable" rather than "impede" the 
development of creative educational approaches that will facilitate successful 
outcomes and still preserve due process rights for students with disabilities. 

• Accountability for Outcomes. Accountability for the implementation of the IDEA has, 
arguably, been limited to monitoring procedural compliance and Federal data 
reporting. Many stakeholders have expressed a need for information focusing on 
defining and assessing learning outcomes for students with disabilities. 

• Community Supported Schools. Given the changing demographic conditions of 
American society, and the concomitant diversity of students' needs, stakeholders have 
expressed the need for information on how the school can become a focal point for 
the delivery of comprehensive services from other community based agencies to 
students and their families. 

• Personnel Development. For real and substantial change in our educational system 
to take place, training and ongoing retraining of teachers is clearly essential. 
Stakeholders have expressed the need for information regarding how to reconfigure 
the relationships and responsibilities of staff and redesign programs that train both 
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special and regular education personnel to work collaboratively in school 
environments. 

Interagency Collaboration. Stakeholders report that fragmentation in the delivery of 
services from multiple agencies continues to be problematic. Stakeholders further 
report the need for information about a consistent system of service delivery based 
on well managed tracking systems that support wrap-around and follow-along service 
models that will reduce gaps in services and lessen burdens on families. 

Technolog^ud Capacity. According to stakeholders, the accelerating evolution of 
technology poses unique challenges for school systems. Changes in technology will 
impact on the information needed for program improvement and on the development 
and analysis of policy affecting all aspects of instruction, administration, and 
evaluation. 

Memative Assessments for EUgibUity. Considerable controversy currently exists over 
the process of determining eligibility for special education. Information is needed 
about how to best conduct assessments that are reliable, valid, cost effective, and 
useful for eligibility determination as well as lEP development, program design, and 
outcome evaluation. 

Over-represerttation of Minorities in Special Edm^on. Information is needed about 
determining participation of minority students in special education. Stakeholders are 
interested in knowing why minority students are disproportionately represented in 
special education as well what kinds of actions are effective in changing these 
patterns. 

Children on the "Boundary". It is currently recognized that many at-risk children are 
"falling through the cracks" between regular and special education. Sometimes these 
children are considered socially maladjusted or ADD; sometimes these children may 
be eligible under Section 504 but not under the IDEA. Stakeholders need 
information to develop policy that fairly addresses these students' needs. 

The Role of Special Education in General Education Reform. The systemic reform of 
U.S. public schools remains a high priority on the Federal agenda for education and 
in many States. To what degree should students with disabilities be included in 
school and Statewide testing programs? To what extent should the abilities of 
students with disabilities be considered in the development of educational standards? 
What implications does site-based management have for special educators? 
Information is needed relative to these questions and how general education reform 
efforts are related to the development of unified or inclusive approaches to the 
education of students with disabilities. 
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• AuxUiary Personnel. There are predicted shortages in related services personnel such 
as occupational and physical therapists, and school psychologists. In addition, it is 
predicted that paraprofessionals will have an increasingly important role in service 
delivery. Issues in this category include personnel demand and supply, the effective 
use of personnel, the training of professional and paraprofessional personnel, and 
cost-efficient ways of providing related services. 

Delphi Process 

A two-round Delphi process was used to rank the relative importance of the broad 
categories of national program information needed. For the first round, all members of the 
Polling Panel were asked to react to the initial list of information needs. (See Appendix B 
for materials sent to Polling Panel members for the first round.) Polling Panel members 
were instructed to add items that represented their constituency's critical information needs. 
Edits and revisions to the list were encouraged. Polling Panel members were asked to 
identify the most important information needs as well as indicate those that were not of 
interest to their constituencies. Project FORUM staff analyzed the Polling Panel's feedback, 
synthesized the stakeholder input, and refined the information needs statements. 

During round two Polling Panel members were asked to re-rate the categories of 
information needed relative to the impact of the information need on improving programs 
for students with disabilities and the importance of the information in leveraging program 
improvement in the not too distant future. The ratings from the first round of the Delphi 
process were also included so that Polling Panel members could re-evaluate their ratings 
based on those of other respondents. (See Appendix C for materials sent in the second 
round.) For the second round, panel members were offered the option of providing their 
feedback to a Project FORUM staff member by phone rather than having to complete the 
response form and mail it back. 

Results 

As outlined in the methods section of this report, the Delphi process consisted of two 
rounds. In the first round 17 out of 31 members of the Polling Panel (55 percent) 
responded to the survey. This return rate, while it was disappointing to Project FORUM 
staff, is not inconsistent with the return rate for many other surveys in which there is no 
follow-up letter or phone call^ The return rate does, however, indicate that caution should 
be exercised in generalizing from the results. Follow-up phone calls were initiated for the 
second round of the process in order to improve the return rate. 



^Sce Borg and Gal! (1983), Educational Research: An Introduction , pp. 429^34, for a discussion of expected return rates and the 
effects of follow-up strategies. 
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A summary of their responses for round one is shoviv in Table 1. Respondents 
tended to give all categories a high priority rating. The categories of Accountability for 
Outcomes, Personnel Development, and Role of Special Education in Reform \received the 
most high priority ratings (15 or 88 percent of the 17 respondents). The category of Over- 
representation of Minority Students in Special Education received the least high priority ratings 
(7 or 41 percent). 

Although the survey was structured to elicit definite responses as to whether a 
category was a high or low priority, the categories of Community Supported Schools, 
Alternative Assessments for Eligibility, and Auxiliary Personnel each received one medium 
priority rating. 

The category of Technological Capacity received the most low priority ratings (7 or 
41 percent). The category of Over-representation of Minority Students in Special Education 
received the most not needed (3 or 18 percent). No other information needs category 
received more than one rating of not needed. 

Respondents did not rate some categories. However, no category had missing data 
from more than two respondents. In all, there were 14 missing responses. 

Conmients from the Polling Panel members tended to underscore the intensity with 
which particular ratings were assigned (e.g., "Yes, this category is a real need in my area.-). 
Some respondents suggested that it was difficult imt to rate all categories as a high priority. 
Difficulty in rating the information needs categories was attributed to their being very broad 
and over arching. Several respondents suggested adding a category regarding general and 
special education finance and funding and reimbursement issues. (Advice that was followed 
in the second round of the Delphi process.) One conmienter felt that the language of the 
survey was quite abstruse and that the categories encompassed many information needs, thus 
making it difficult to give one, overall rating to a category. This commenter stated that the 
survey needed to be redesigned and that the best approach was to "start over." 

The return rated increased in the second round with 20 out of 31 Polling Panel 
members (65%) responding. For round two of the Delphi process, members of the Polling 
Panel were sent a new survey that included the new category of finance along with 
respondents' ratings from round one of the process. Members of the Polling Panel were 
asked to reconsider their ratings based on the ratings of other respondents, and to edit, 
revise, and provide additional conmients. Polling Panel members were contacted by 
telephone so that they could provide their ratings and comments verbally if they so desired. 

The results from round two of the process are shown in Table 2. They are consistent 
with those of round one. Respondents rated most categories as high priority. The categories 
of Accountability for Outcomes, Personnel Development, and Role of Special Education in 
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Table 1 

Responses for Round One 
n=17 



Categories 


Hioh nrinritv 


T f\w tinonfv 




i^u laiUlK 


^ccuuniaoiuty lor vjUiConies 


Ij (00%) 


1 (0%) 


u 


1 (6%) 


i^cibunnei ueveiopineni 


Id (00%) 


1 (0%) 


0 


1 (6%) 


ivuk; ui opcciai iiaiicuiion in Keionn 


15 (00%) 


0 


1 (6%) 


1 (6%) 


Auxiliary Personnel^ 


13 (76%) 


3 (18%) 


0 


0 


Interagenqr Collaboration 


13 (76%) 


3 (18%) 


0 


1 (6%) 


Alternative Assessments^ 


11 (65%) 


4(23%) 


0 


1 (6%) 


Program Management 


11 (65%) 




1 (6%) 


1 (6%) 


Children on the "Boundary" 


10 (60%) 


4(24%) 


1(6%) 


2(12%) 


Community Supported Schools^ 


10 (58%) 


3 (18%) 


1(6%) 


2(12%) 


Technological Capacity 


8 (47%) 


7 (41%) 


0 


2(12%) 


Over-representation of Minorities 


7 (41%) 


5(30%) 


3 (18%) 


2(12%) 



^ One respondent rated this category as a "medium priority." 

Reform again received the most high priority ratings. (In fact. Personnel Development^ and 
Role of Special Education in Reform received only high priority ratings.) As in the first 
round, Technological Capacity and Over-representation of Minorities received the most low 
priority ratings. The new category of Finance/Funding Formulas received 12 (60 percent) 
high priority ratings. However, it also received three (15 percent) no ratings, and comments 
indicated that this item would have been more highly rated if the category had not included 
the term "reform" reading simply "general and special education finance." 
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Table 2 

Responses for Round Two 
n=20 



Categories 


High priority 


Low priority 


Not needed 


No rating | 


Personnel Development 


20 (100%) 


0 


0 


0 


Role of Special Education in Reform 


20 (100%) 


0 


0 


0 


Accountabilit}' for Outcomes 


19 (95%) 


1 (5%) 


0 


0 


Interagency Collaboration 


17 (83%) 


3(15%) 


0 


0 


Auxiliary Personnel 


13 (65%) 


6(30%) 


0 


1(6%) 


Program Management 


13 (65%) 


6 (30%) 


1 yj/o) 


U 


Aiternative Assessments^ 


12(56%) 


5 (25%) 


0 


2(10%) 


Rnance/Funding Formulas^ 


12 (60%) 


4(20%) 


0 


3(15%) 


Children on the "Boundary" 


11 (55%) 


8 (40%) 


0 


1 (5%) 


Community Supported Schools^*^*^ 


11 (55%) 


5(25%) 


0 


0 


Over-representation of Minorities^ 


8 (40%) 


8(40%) 


1 (5%) 


1(5%) 


Technological Capadty^ 


6(30%) 


12(60%) 


1 (5%) 


0 



Note: Percentages may not add up to 100 percent due to rounding. 

^ Received '*medium priority" rating(s) 
^ One respondent rated this as '^other" 
^ One respondent rated this as "high in cities, low in suburbs'* 



Prioritized List 



A prioritized list of the information needs categories resulting from this process is 
presented below. Information needs with identical ratings by the Polling Panel are listed 
alphabetically. 

Personnel Development, (1(X) percent) 

The Role of Special Education in General Education Reform. (100 percent) 
Accountability for Outcomes, (95 percent) 
Interagency Collaboration, (85 percent) 
Auxiliary Personnel, (65 percent) 
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• Program Management and Regulatory Flexibility. (65 percent) 

• Alternative Assessments for Eligibility. (60 percent) 

• General and Special Education Finance Reform. (60 percent) 

• Children on the "Boundary". ' (55 percent) 

• Community Supported Schools. (55 percent) 

• Over-representation of Minorities in Special Education. (40 percent) 

• Technological Capacity. (30 percent) 

Conclusions 

The purpose of this activity was to utilize a stakeholder panel to identify and 
prioritize information needed to improve the management, administration, delivery and 
effectiveness of special education programs and present the results to OSEP for use in its 
program planning. Several considerations for OSEP did emerge and these are discussed 
below. However, the limitations of the method used to obtain the prioritized list of 
information needs should be noted and generalizations and decisions based on it should be 
made cautiously. 

Lessons Learned 

While the Polling Panel approach chosen by Project FORUM was a realistic one 
given the limited resources available for the task, the individuals participating did not 
represent a random sample of national stakeholders. Nevertheless, the range panel member 
responses regarding the relative importance of even such broad categories of information 
need as described in our survey probably does represent the diversity of conditions and 
circumstances experienced by stakeholders located at different points in the system. 

Because some respondents commented that "they would rank them all high," actual 
ratings very likely reflects the relative position of the categories rather than an absolute 
standard. The rt. jlts of this process, therefore, should not be used to eliminate any 
category from further consideration as a potential area of information need. 

In addition, the process did not necessarily reflect the intensity of respondents' ratings. 
Future surveys should employ a Likert scale ranking to allow for finer distinctions among 
opinions of information needs. 

The results of the process of initial implementation of the plan are probably best 
viewed as tentative and as part of an evolving process to more systematically and accurately 
determine the major categories of national information needed for program improvement. 

Project FORUM and NASDSE are committed to providing OSEP with the best 
possible information and therefore intend to apply the "lessons learned" from this activity 
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to future efforts. Among these are: continuing to build upon the State-level needs 
identification efforts of other OSEP sponsored activities and projects (e.g., the Regional 
Resources Centers; the National Information Action Center at NASDSE); obtaining 
information about the needs of other stakeholder groups directly from national organizations 
that serve these constituencies; reviewing the role of the Polling Panel particularly with 
regard to evaluating the initial list of information needs; and refining the survey process and 
instrument used to establish the relative priority of information needs. 

Recommendations 

The results of the process did, however, yield some indication of the particular 
management, administration, and service delivery strategies needing improvement. Areas 
that respondents clearly identified as priorities were Accountability for Outcomes, Personnel 
Development, and Role of Special Education in Reform. The high priority given to these 
clusters appears to represent high degree of need for information that can guide State and 
local decisionmakers responsible for implementing policies and practices that will improve 
the results of education for children and youth with disabilities. OSEP has a number of 
investments in projects that are generating information that can address these needs. 
Among these are the National Center on Educational Outcomes (NCEO), the Center for 
Policy Studies at the University of Maryland, and the Center for Special Education Finance 
(CSEF). The National Longitudinal Transition Study (NLTS) being conducted for OSEP 
by SRI, International has provided a wealth of information about the post school outcomes 
of students with disabilities and identified a number of variables that can influence the 
direction of these outcomes. However, the integration of information available from these 
and other OSEP research and information projects by State and local special education 
decisionmakers has not been fully realized. 

The results of this initial effort at identifying information needed to improve the 
management, administration, delivery, and effectiveness of special education programs offer 
several suggestions for OSEP's consideration. One of these is investing in activities that will 
assist OSEP's network of knowledge producers to develop vehicles for collaborating with 
local administrators and teachers in order to produce information that is relevant to their 
current information needs. A second strategy is to facilitate the timely and meaningful 
exchange of information between OSEP's knowledge produces and the State and local 
administrators who are in a position to apply this knowledge to improving the education 
programs serving infants, toddlers, children, and youth with disabilities. 
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APPENDIX A . Ust of Polling Panel Members 



NAME 



AFHLIATION 



Alan Abeson 
Amie Amiot 

Fred Balcom 

Amy Batiste 

Frank Betts 

Lynne Cook 

Smokey Davis 
Harold Dent 
Grace Zamora Duran 
Sue Ganmi 
Rhona Hartman 
Barbara Huff 

Judy Katz-Leavy 

Ed Keller 

Chris Koyanagi 
Justine Maloney 
Paul Marchand 
Gary Marx 

Jonathan Mclntire 



The ARC 

American Speech-Language-Hearing 
Association 

Idaho State Department of 
Education 

Clearinghouse on Professions in 
Special Education 

Association for Supervision and 
Curriculum Development 

Clearinghouse on Professions in 
Special Education 

Associate Director, NASDSE 

Hampton University 

Council for Exceptional Children 

Chicago Public Schools 

HEATH Resource Center 

Federation of Families for 
Children's Mental Health 

Child and Family Support Branch 
CASSP, NIMH 

National Association of Elementary 
School Principals 

National Mental Health Association 

Learning Disabilities Association 

The ARC 

American Association of School 
Administrators 

Rutland (VT) SW Supervisory 
Union 



Celane McWhorter 

Ruth Mondschein 
Joan Nelson 
Suzanne Ripley 

Virginia Roach 

Jeff Schneider 
Damans Sifuentes 

Julian Taplin 



The Association for Persons with 
Severe Handicaps 

LaRaza 

NIAC at NASDSE 

National Information Center for 
Children & Youth with Handicaps 

National Association of State Boards 
of Education 

National Education Association 

Puerto Rico Department of 
Education 

Division of Child Mental Health 
Services (DE) 



Jo Thomason 

Frederick Weintraub 

Judith Weitz 
Martha Ziegler 



Council of Administrators of Special 
Education 

The Council for Exceptional 
Children 

Center for the Study of Social Policy 

Federation for Children with 
Special Needs 



ERIC 
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^^■^fiOff I National Association of State Directors of Special Education, Inc 



>:^^^,^ jr xei, 703/519-3800 • Fox: 703/519-3808 • TDD: 703/519-7008 • SpecialNet: NASDSE 



King Street Station I, 1800 Diagonal Road, Suite 320, Alexandria, VA 22314 



April 16, 1993 

1-^ 
2^ 

Dear 3^: 

Thank you for agreeing to being on Project FORUM's polling panel and participating in our efforts to 
determine the information needed to improve the management, administration, delivery and effectiveness 
of special education programs for infants, toddlers, children, and youth with disabilities. As you know 
from conversations with Ed McCaul, this is the first phase of a multi-year activity. Input from the full 
spectrum of stakeholders in the special education community is very important to the success of this 
erdeavor. 

Enclosed you will find a brief background paper explaining the task and providing a framework for the 
task you have agreed to perform. The list of categories of information needs is also enclosed. After 
examining this material, please evaluate the information needs categories relative to the following: 



• Indicate which of these needs are a high priority, a low priority or do not reflect the 
information needs of your constituents. 

• Revise or edit information needs to clarify or more accurately state the questions or issue. 

• Add other information needs that are important to your constituents or colleagues. 

Please make any additional comments you think would improve this process. After Project FORUM staff 
have analyzed the responses from the polling panel, we will be mailing the revised list to you for 
additional verification and prioritization of the categories. 

Once again, we thank you for participating in this effort. If you have any questions, please call me or 
Ed McCaul at 703-519-3800. 



Sincerely, 



Trina W. Osher, Director 
Project FORUM 



cc: Smokey Davis, Associate Director 



A PLAN FOR THE 
IDENTIFICATION OF NATIONAL INFORMATION 
NEEDED FOR PROGRAM IMPROVEMENT 

Prepared by Project FORUM at NASDSE 
Under U.S. ED Contract No. HS92015001 



Background and Introduction 

The importance of information in decision making and policy development has long been 
recognized, and contemporary technology has made the identification, tracking, and analysis of 
information in a timely fashion a critical necessity for organizations in our society. It is 
impossible for individuals who are responsible for the implementation of the Individuals with 
Disabilities Education Act (IDEA) to access and read the ever-giowing array of information that 
could be useful in making decisions that would improve the system and yield better results for 
infents, toddlers, children, and youth with disabilities. Recognizing the implications of this 
situation prompted Congress to amend Section 618(c)(3) of the IDEA to assure that information 
would be gathered and analyzed that was "necessary for achieving program and system 
improvements at the State and local levels". 

The statute directs the Secretary of Education to develop and implement "a propess for 
the on-going identification of national program information needed for improving the 
management, administration, delivery and effectiveness of programs and services under the Act." 
The process is to be conducted in cooperation with State Education Agencies (SEAs) to insure 
broad statewide input. The National Association of State Directors of Special Education 
(NASDSE) under its Project FORUM Contract No. HS92015001 with the Office of Special 
Education Programs (OSEP) has the task of assisting OSEP with this effort. This paper presents 
an overview of Project FORUM's plan to establish and implement the on-going process called 
for in the Act. 

During December 1992, Project FORUM convened a series of three stakeholder meetings 
to obtain input into the design of the information needs identification process. Participants were 
given a briefing paper outlining the task before attending the informal half-day meetings. 
Telephone interviews were conducted with stakeholders who could not attend any of the 
meetings. In all, 19 stakeholders were contacted, 13 attended the stakeholder meetings, and 3 
were interviewed by telephone. 



The Plan 

After preliminary work with a consultant with expertise in the area of needs analysis. 
Project FORUM staff chose to utilize a marketing model for information need identification and 
analysis. Stakeholders who had attended planning meetings reviewed the plan and supported the 
use of this model. They particularly favored the concept of stakeholders as "customers." The 
marketing model involves polling "real" stakeholders, not just "experts", to find out what they 



perceive to be their information needs. This approach is time and cost effective if the 
stakeholders have an interest in the outcome of the process and the information bemg requested. 
The first step in this process involves the establishment of a "polling panel" of interested and 
committed stakeholders. 

Creation of a polling panel. The marketing model relies on using a group or groups of 
consumers as informants. For the purpose of identifying national information needed for 
improving the management, administration, delivery, and effectiveness of programs and services 
for infants, toddlers, children, and youth with disabilities. Project FORUM is establishing a 
polling panel to function in this role. We propose to use the polling panel in several ways The 
full panel will be asked to react to and revise working lists of potential information needs tor 
program improvement. Polling panel members will also be involved in verifying and pnontizing 
the lists as well as collapsing the lists into broader categories of information. Finally, we expect 
to involve polling panel members in reassessing the process for identifying national program 
information needs on an annual basis. 

Creating an initial list of national program information needs. The marketing model 
requires that consumers react to a 'test product'. In this task, the 'test product' is an initial hst 
of national program information needs. There are an unlimited number of ways of generatmg 
such a list ranging from pure speculation to elaborate scientific surveying. Given the relatively 
urgent need OSEP has for the information and in keeping with the advice of stakeholders. Project 
FORUM has derived this list from existing information sources. 

For the first year. Project FORUM is asking members of the polling panel to respond to 
a list that includes information needs generated in January of 1992. At that time. Project 
FORUM convened a two-day Focus Group of experts to assist OSEP in identifying information 
needed for program improvemem un^er Section 618 of the IDEA. Other sources used to 
generate the list for Year I include information from Project FORUM's Issues Tracking Database 
results of the annual survey of State Directors of Special Education (conducted by the National 
Information Action Center at NASDSE), and similar material supplied by other national 
associations such as NASBE and CCSSO. Also, national clearinghouses, such as NICHCY and 
ERIC at CEC, maintain databases about the inquiries they receive and are sharing some of this 
information with Project FORUM fro consideration as possible topics to include on this list of 
national information needs. A copy of the list of information needs is attached at the end of this 
paper. 

Project FORUM is asking all members of the polling panel to react to the list of 
information needs. Polling panel members should add items they believe represent their 
constituency's critical information needs. Edits and revisions to the list are encouraged. Polling 
panel members are also asked to identify the most important information needs and indicate those 
that are not of interest to their peers. 
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Project FORUM staff will analyze the polling panePs feedback and reorganize the list of 
identified needs by placing similar and related items into broad categories and synthesizing the 
stakeholder input for each category into a single but comprehensive information need statement. 
Project FORUM staff may use a conference call strategy to involve several members of the 
polling panel to clarify issues raised and to discuss options for organizing the list concisely. 

Prioritizing the categories of identified needs. After reorganizing the list, Project 
FORUM staff will employ a Delphi process to rank the relative importance of the broad 
categories of national program information needed. At this stage, polling panel members will 
be asked to rank, on a scale of 1 (low) to 5 (high), the categories of information needed relative 
to both the impact of the information need on improving programs for students with disabilities, 
and the likelihood of obtaining sufficient information to leverage program improvement in the 
not too distant future. 

Results. Using feedback from the Delphi process, information needs will be ranked 
according to stakeholder ratings, and the results provided to OSEP for use in its program 
planning. Project FORUM will also make the results available to all the stakeholder groups 
participating in the process and other interested parties. Project FORUM will use the results of 
this process by organizing a wide variety of information for potential use in achieving program 
improvements. 

Time Line - Year 1 



April Initial list of information needs sent to polling panel 

May Feedback analyzed and synthesized into categories - possible conference 
calls with panel members for clarification and further validation 

June Delphi process for prioritization conducted 

July Delphi results analyzed and final list prepared 

August Report to OSEP on results of process 

September Final list published 



Throughout the process, polling panel members will be asked for feedback regarding both the 
results and the process utilized for this first year. The active involvement of polling panel 
members is necessary to the success of the overall effort to identify and track the information 
needed for program improvement. 
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A PLAN FOR THE IDENTIFICATION OF 
NATIONAL INFORMATION NEEDED FOR PROGRAM IMPROVEMENT 

Project FORUM - Year I 
US. ED Contract No. HS92015001 
April 16, 1993 

Categories of National Program Information Needs 



Program Management and Regulatory Flexibility. Currently, many stakeholders express 
interest in information about how regulations may "enable" rather than "impede" the 
development of creative educational approaches that will facilitate successful outcomes 
and still preserve due process rights for students with disabilities. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



• Accountability for Outcomes. Accountability for the implementation of the IDEA has, 
arguably, been limited to monitoring procedural compliance and Federal data reporting. 
Many stakeholders have expressed a need for information focusing on defining and 
assessing learning outcomes for students with diabilities. 

High Priority Low Priority Not Needed 



Comments, edits, and suggestions: 



Community Supported Schools. Given the changing demographic conditions of 
American society, and the concomitant diversity of students' needs, stakeholders have 
expressed the need for information on how the school can become a focal point for the 
delivery of comprehensive services- from other community based agencies to students and 
their families. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



Personnel Development. For real and substantial change in our educational system to 
take place, training and ongoing retraining of teachers is cleariy essential. Stakeholders 
have expressed the need for information regarding how to reconfigure the relationships 
and responsibilities of staff and redesign programs that train both special and regular 
education personnel to work collaboratively in school environments. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 
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Interagency Collaboration. Stakeholders report that fragmentation in the delivery of 
services from multiple agencies continues to be problematic. Stakeholders further report 
the need for information about a consistent system of service delivery based on well 
managed tracking systems that support wrap-around and follow-along service models that 
will reduce gaps in services and lessen burdens on families. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



Technological Capacity. According to stakeholders, the accelerating evolution of 
technology poses unique challenges for school systems. Changes in technology will 
impact on the information needed for program improvement and on the development and 
analysis of policy affecting all aspects of instmction, administration, and evaluation. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



ERIC 
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AUemative assessments for eligibility. Considerable controversy currently exists over the 
process of determining eligibility for special education. Information is needed about how 
to best conduct assessments that are reliable, valid, cost effective, and useful for eligibility 
determination as well as lEP development, program design, and outcome evaluation. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



Over representation of minorities in special education. Information is needed about 
determining participation of minority students in special education. Stakeholders are 
interested in knowing why minority students are disproportionately represented in special 
education as well what kinds of actions are effective in changing these patterns.. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 
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Children on the "boundary". It is cunently recognized that many at-risk children are 
"falling through the cracks" between regular and special education. Sometimes these 
children are considered socially maladjusted or ADD; sometimes these children may be 
eligible under Section 504 but not under the IDEA. Stakeholders need information to 
develop policy that fairly addresses these students' needs. 

High-priority Low Priority Not Needed 

Comments, edits, and suggestions: ■ 



The role of special education in general education reform. The systemic reform of U.S. 
public schools remains a high priority on the Federal agenda for education and in many 
States. To what degree should students with disabilities be included in school and 
Statewide testing programs? To what extent should the abilities of students with 
disabilities be considered in the development of educational standards? What implications 
does site-based management have for special educators? Information is. needed relative 
to these questions and how general education reform efforts are related to the 
development of unified or inclusive approaches to the education of students with 
disabilities. 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 
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Auxiliary personnel. There are predicted shortages in related services personnel such as 
occupational and physical therapists, and school psychologists. In addition, it is predicted 
that paraprofessionals will have an increasingly important role in service delivery. Issues 
in this category include personnel demand and supply, the ef fective use of personnel, the 
training of professional and paraprofessional personnel, and cost-efficient ways ot 
providing related sewices. 



High Priority 



Low Priority Not Needed 



Comments, edits, and suggestions: 



ADDITIONAL CATEGORIES/COMMENTS: Attach additional sheets if necessary. 



THANKS FOR YOUR HELP 

Please use the enclosed envelope to return your feedback to: 

Edward McCaul 
Project FORUM at NASDSE 
1800 Diagonal Road, Suite 320 
Alexandria, VA 22314 
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APPENDIX C - Materials sent to Polling Panel for Second Round of the Delphi Process 



rji\gnnp \ National Association of State Directors of Special Education, Inc. 

iVrU^ ^ King street station i^jmOiagoiml Road, Suite 3^^^ VA 22314 

%3>^ o-,'^ Teb 703/519-3800 FAX: 703/519-3808 TDD: 703/519-7008 SpedalNet NASDSE 

June 16, 1993 

2^ 

Dear 3 ^ : 

Thank you for your response to our survey on the information needed to improve the management, 
administration, delivery and effectiveness of special education programs for infants, toddlers, children, and 
youth with disabilities. As you know from conversations with Ed McCaul, this is the first phase of a 
multi-year activity. Input from the full spectrum of stakeholders in the special education community is 
very important to the success of this endeavor. 

Enclosed b a revised list of information needs based on the responses to our first mailing. For those of 
you that responded to our first mailing, this provides you with the opportunity to revise and prioritize 
your ratings based on the ratings of other respondents. For those of you who did not respond, this 
provides you with the opportunity to rate the categories and give us feedback on the information needs 
of your constituents. The document shows the percentage of respondents rating each category as high 
priori tv. low priority, or not needed, and provides space for your new rating, comments, edits, and 
suggestions. 



Please consider how to re-evaluate the information needs categories relative to the following: 

• Indicate which of these needs are a high priority, a low priority or do not reflect the 
information needs of your constituents. 

• Revise or edit information needs to clarify or more accurately state the questions or issue. 

• Add other information needs that are important to your constituents or colleagues. 

In order to expedite this second round of the process, Ed McCaul will be contacting you by phone shortly 
after June 17'**. He will be prepared to take your new ratings verbally and transcribe your comments. 
If you prefer to make your own written response, please write directly on the enclosed document and 
return it to Ed McCaul, Project FORUM at NASDSE, 1800 Diagonal Road, Suite 320, Alexandria, VA 
22314 as soon as possible and no later than June 22 "'I 

After Project FORUM staff have analyzed the second round of responses, we will be sending you a 
summary of the results. We thank you for participating in this effort. If you have any questions, please 
call me or Ed McCaul at 703-519-3800. 



Sincerely, 



Trina W. Osher, Director 
Project FORUM 



cc: Dr. Martha J. Fields, Executive Director 



A PLAN FOR THE IDENTIFICATION OF „„.,„^^ 
NATIONAL INFORMATION NEEDED FOR PROGRAM IMPROVEMENT 

Project FORUM - Year I 
\5S. ED Contract No. HS92015001 
June 11, 1993 

Categories of National Program Information Needs 

PragnimM«iwg«m«irf«iidK«g«iaio/yF/e«6i%. Currently, ma 
interest in information about how regulations may "enable" rather than impede the 
development of creative educational approaches that will facilitate successful outcomes 
and still preserve due process rights for students with disabilities. 



Ratings of other respondents: 

High Priority -63% Low Priority -25% 

Note: Some respondents did not rate this category. 
Your new rating: 

High Priority Low Priority 



Not Needed -6% 



Not Needed 



Comments, edits, and suggestions: 



Accountability for Outcomes. Accountability for the implementation of the IDEA has, 
arguably, been limited to monitoring procedural compliance and Federal data reporting. 
Many stakeholders have expressed a need for information focusing on defimng and 
assessing learning outcomes for students with disabilities. 



Ratings of other respondents: 

High Priority -88% Low Priority -6% 

Note: Some respondents did not rate this category. 
Your new rating: 

High Priority Low Priority 



Not Needed -0% 



Not Needed 



Comments, edits, and suggestions: 



Community Supported Schools. Given the changing demographic conditions of 
American society, and the concomitant diversity of students' needs, stakeholders have 
expressed the need for information on how the school can become a focal point for the 
delivery of comprehensive services from other community based agencies to students and 
their families. 



Ratings of other respondents: 
High Priority -56% 



Low Priority -19% 



Not Needed-6% 



Note: Some respondents did not rate this category; one respondent rated it as a "medium 
priority." 



Your new rating: 
High Priority 



Low Priority 



Not Needed 



CommentSy edits, and suggestions: 



Personnel Development. For real and substantial change in our educational system to 
take place, training and ongoing retraining of teachers is cleariy essential. Stakeholders 
have expressed the need for information regarding how to reconfigure the relationships 
and responsibilities of staff and redesign programs that train both special and regular 
education personnel to work collaboratively in school environments. 



Ratings of other respondents: 

High Priority -88% Low Priority -6% 

Note: Some respondents did not rate this category. 
Your new rating: 



High Priority 



Low Priority 



Comments^ edits, and suggestions: 



Not Needed-0% 



Not Needed 
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Interagency Collaboration. Stakeholders report that fragmentation in the delivery of 
services from multiple agencies continues to be problematic. Stakeholders further report 
the need for information about a consistent system of service delivery based on well 
managed tracking systems that support wrap-around and follow-along service models that 
will reduce gaps in services and lessen burdens on families. 

Ratings of other respondents: 

High Priority -75% Low Priority -19% Not Needed -0% 

Note: Some respondents did not rate this category. 

Your new rating: 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



• Technological Capacity. According to stakeholders, the accelerating evolution of 
technology poses unique challenges for school systems. Changes in technology will 
impact on the information needed for program improvement and on the development and 
analysis of policy affecting all aspects of instruction, administration, and evaluation. 

Ratings of other respondents: 

High Priority -44% Low Priority -44% Not Needed-0% 

Note: Some respondents did not rate this category. 
Your new rating: 

High Priority Low Priority Not Needed 



Comments, edits, and suggestions: 



• Alternative assessments for eligibility. Considerable controversy currently exists over the 
process of determining eligibility for special education. Information is needed about how 
to best conduct assessments that are reliable, valid, cost effective, and useful for eligibility 
determination as well as lEP development, program design, and outcome evaluation. 

Ratings of other respondents: 

High Priority -63% Low Priority -25% Not Needed -0% 

Note: One respondent rated this category as a "medium priority." 
Your new rating: 

High Priority Low Priority Not Needed 

CommentSy editSy and suggestions: 



• Over representation of minorities in special education. .Information is needed about 
determining participation of minority students in special education. Stakeholders are 
interested in knowing why minority students are disproportionately represented in special 
education as well what kinds of actions are effective in changing these patterns. 

Ratings of other respondents: 

High Priority -38% Low Priority -31% Not Needed -19% 

Note: Some respondents did not rate this category. 
Your new rating: 

High Priority Low Priority Not Needed 

CommentSy editSy and suggestions: 



i 
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• Children on the "boundary". It is currently recognized that many at-risk children are 
"falling through the cracks" between regular and special education. Sometimes these 
children are considered socially maladjusted or ADD; sometimes these children may be 
eligible under Section 504 but not under the IDEA. Stakeholders need information to 
develop policy that fairly addresses these students' needs. 

Ratings of other respondents: 

High Priority -56% Low Priority -25% Not Needed -6% 

Note: Some respondents did not rate this category. 
Your new rating: 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 



• The role of special education in general education reform. The systemic reform of U.S. 
public schools remains a high priority on the Federal agenda for education and in many 
States. To what degree should students with disabilities be included in school and 
Statewide testing programs? To what extent should the abilities of students with 
disabilities be considered in the development of educational standards? What implications 
does site-based management have for special educators? Information is needed relative 
to these questions and how general education reform efforts are related to the 
development of unified or inclusive approaches to the education of students with 
disabilities. 

Ratings of other respondents: 

High Priority -88% Low Priority -0% Not Needed -6% 

Note: Some respondents did not rate this category. 
Your new rating. 

High Priority Low Priority Not Needed 



Comments, edits, and suggestions: 



• Auxiliary personnel There are predicted shortages in related services personnel such as 
occupational and physical therapists, and school psychologists. In addition, it is predicted 
that paraprofessionals will have an increasingly important role in service delivery. Issues 
in this category include personnel demand and supply, the effective use of personnel, the 
training of professional and paraprofessional personnel, and cost-efficient ways of 
providing related services. 

Ratings of other respondents: 

High Priority -75% Low Priority -19% Not Needed-0% 

Note: One respondent rated this category as a "medium priority." 
Your new rating: 

High Priority Low Priority Not Needed 

Comments, editSy and suggestions: . 



• Several respondents suggested a category on general and special education finance 
reform. I would rate this category as: 

High Priority Low Priority Not Needed 

Comments, edits, and suggestions: 
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ADDITIONAL CATEGORIES/COMMENTS: Attach additional sheets if necessary. 



THANKS FOR YOUR HELP 

Please use the enclosed envelope to return your feedback by June 14 to: 



Edward McCaul 
Project FORUM at NASDSE 
1800 Diagonal Road, Suite 320 
Alexandria, VA 22314 
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